In mobile radio communications system, the quality of service provided to subscribers is very important. What is important in voice and video services is the perception by a human being of the quality of a voice message or a video message. One way to measure that quality is to conduct subjective tests involving various human beings. But the results could be individually subjective, and the tests are expensive and unsuitable for large scale quality monitoring.
There are several algorithms developed to measure the “subjective” speech quality of a transmitted voice file. One example is the Perceptual Evaluation of Speech Quality (PESQ) algorithm defined in ITU-T P.862. The subjective quality is not measured based upon conventional radio channel quality measurement methods such as signal-to-voice ratio (SIR), a bit error rate (BER), frame error rate (FER), or signal strength. Rather, PESQ and similar algorithms predict the results of subjective listening tests. To measure speech quality, PESQ uses a sensory model to compare an original, unprocessed, untransmitted speech signal with a degraded version of that known speech signal at the output of the communications system, e.g., the signal received by a radio receiver after having been distorted by a radio channel. Comparing the reference and degraded speech signals provides subjective quality score.
Subjective quality measurement algorithms, like PESQ, are resource intensive requiring large amounts of data processing calculations and considerable memory resources. This large data processing load is further exacerbated if quality measurements and calculations are made frequently or even continuously. Accordingly, mobile test units (MTUs) include substantial data processing hardware and software necessary to calculate and store measurement data associated with subjective quality determination algorithms like the PESQ algorithm. Video quality measurement algorithms are presumably even more resource demanding. These MTU vehicles collect measurement data under realistic conditions with respect to the existing radio conditions at various locations in the mobile radio communications system.
But these MTUs are very costly, heavy, and consume considerable amounts of power, making them unsuitable for a large-scale measurement where quality measurements are taken frequently from a large number of measurement units. While it might be feasible in a commercial setting to employ several MTUs, it would be much more desirable to make the measurements on a much larger scale, perhaps, on the order of hundreds, thousands, or tens of thousands of measurement units. In the future, there may be a need to include Quality Measurement Functionality in every commercial user equipment so that an operator may start and stop the measurements based on time, location, or type of problem. But this large scale is costly, not only in terms of the large number of expensive MTUs, but also in terms of the human resources required to locate and/or operate those MTUs.
The present invention overcomes these obstacles and achieves these and other desirable goals. Large scale subjective signal quality measurements for a mobile radio communications system are obtained using a large number of handheld subscriber radio communication units located at various positions in the mobile radio communications system. Each handheld subscriber unit stores a copy of a test voice, other audio, or video signal stream, as does a quality management network node. An uplink subjective signal quality for each handheld subscriber unit is determined at the network node based on a comparison of the stored test signal and a received test signal transmitted from the handheld subscriber unit. A downlink subjective signal quality to each handheld unit is determined at the network node based on the stored test signal and a test signal stream originally transmitted to the handheld subscriber unit and then returned by the handheld subscriber unit to the network node. Because the handheld units do not perform the demanding subjective signal quality comparison calculations, ordinary handheld subscriber units can be used.
The quality management network node stores multiple uplink quality values and downlink subjective quality values associated with each handheld subscriber unit from which an overall subjective quality associated with each handheld subscriber unit is determined. A location or area in the mobile radio communications system and/or a particular time frame may also be determined for various subjective quality measurements. In one example embodiment, the test signal stream is a voice signal, and the quality associated with each handheld subscriber unit is determined using a Perceptual Evaluation of Speech Quality (PESQ) algorithm. The test stream could also be video or other audio analyzed by other suitable subjective quality determination algorithms.
A database stores a signal quality measurement, a time associated with the subjective signal quality measurement, and a geographic position associated with the subjective signal quality measurement. A network operator provides subjective service quality information associated with various time(s) and location(s) in the mobile radio communications system based on information stored in that database.
In an alternative example embodiment, a handheld subscriber unit receives a test voice, other audio, or video signal, converts that signal into a test data stream, and returns the test data stream to the quality management network node. The quality management node determines a downlink signal quality to the handheld subscriber unit based on the comparison of the stored test signal and the returned test data signal received from the handheld subscriber unit.
In another example embodiment, the speed of handheld subscriber units is taken into an account in the subjective quality determination. A handheld subscriber unit sends its current position and speed of movement to the quality management network node. The quality management network node and the handheld subscriber unit divide the predetermined test signal into portions. The portion size is based on the position and/or speed of the handheld subscriber unit. The quality management network node and the handheld subscriber unit send the test signal stream in multiple portions rather than as one signal stream. Thus, when the handheld subscriber unit returns received portions of the test signal stream to the quality management network node, the network node reassembles the test signal stream from those returned portions. In this or any embodiment, echo cancellation of the returned predetermined test signal is preferably prevented.